Nomad Actions #18794

philrenaud · 2023-10-18T13:52:08Z

Introduces the concept of Actions to Nomad: operator-defined commands and workflows that Nomad users can run on an allocation without having to sweat the details.

These take a similar form to task config blocks and live at task level in the jobspec. To demonstrate:

job "actions-demo" {
  type = "system"

  group "group" {
    task "task" {
      driver = "raw_exec"
      config {
        command = "node"
        args    = ["-e",
        <<EOT
console.log('Howdy!');
// sleep every 3 seconds
setInterval((a) => {
  console.log('Howdy after 3 seconds');
}, 3000);
EOT
        ]
      }

      action "weather" {
        command = "/usr/bin/curl"
        args = ["-s", "wttr.in/Toronto?format=3"]
      }
      action "hello" {
        command = "say"
        args = ["howdy"]
      }
      action "get job info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad job status ${NOMAD_JOB_NAME}
          EOT
        ]
      }
      action "get alloc info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad alloc status ${NOMAD_ALLOC_ID}
          EOT
        ]
      }

      action "echo time" {
        command = "/bin/sh"
        args    = ["-c", "counter=0; while true; do echo \"Running for ${counter} seconds\"; counter=$((counter + 1)); sleep 1; done"]
      }

    }
  }
}

These actions can be run from the CLI with nomad action by passing in either the allocation ID, or if none is provided, the task and group names and an allocation will be randomly selected:

nomad action \
  -allocation=cafee3ae \
  -job=actions-demo \
weather

nomad action \
  -group=group \
  -task=task \
  -job=actions-demo \
"weather"

An endpoint at v1/job/:id/actions will GET return a list of actions the job has access to, and a GET at v1/job/:id/action?action=NAME&allocID=ID&group=GROUP&task=TASK&tty=true will open a websocket with base64-encoded output (be sure to use Upgrade: websocket and related headers)

Future work:

When #18793 lands, the Nomad web UI will display interactive elements to run actions and view their output in various places:

Further, part of the intent behind actions is to give operators greater ability to specify which sorts of things a Nomad user can perform. With this in mind, ACL changes with actions in mind are planned in the near future.

Resolves #18627

* Task-level actions for job submissions and retrieval * FIXME: Temporary workaround to get ember dev server to pass exec through to 4646 * Update api/tasks.go Co-authored-by: Tim Gross <tgross@hashicorp.com> * Update command/agent/job_endpoint.go Co-authored-by: Tim Gross <tgross@hashicorp.com> * Diff and copy implementations * Action structs get their own file, diff updates to behave like our other diffs * Test to observe actions changes in a version update * Tests migrated into structs/diff_test and modified with PR comments in mind * APIActionToSTructsAction now returns a new value * de-comment some plain parts, remove unused action lookup * unused param in action converter --------- Co-authored-by: Tim Gross <tgross@hashicorp.com>

* unused param in action converter * backing out of parse_job level and moved toward new endpoint level * Adds taskName and taskGroupName to actions at job level * Unmodified job mock actions tests * actionless job test * actionless job test * Multi group multi task actions test * HTTP method check for GET, cleaner errors in job_endpoint_test * decomment

github-actions · 2023-10-18T14:12:22Z

Ember Test Audit comparison

	main	`1fe0e11`	change
passes	1533	1534	+1
failures	1	1	0
flaky	0	0	0
duration	000ms	000ms	-000ms

* Working demo for review purposes * removal of cors passthru for websockets * Remove job_endpoint-specific ws handlers and aimed at existing alloc exec handlers instead * PR comments adressed, no need for taskGroup pass, better group and task lookups from alloc * early return in action validate and removed jobid from req args per PR comments * todo removal, we're checking later in the rpc * boolean style change on tty

* Action command init and stuck-notes * Conditional reqpath to aim at Job action endpoint * De-logged * General CLI command cleanup, observe namespace, pass action as string, get random alloc w group adherence * tab and varname cleanup * Remove action param from Allocations().Exec calls

tgross

LGTM once the ACL check comment is fixed.

nomad/job_endpoint.go

kaspergrubbe · 2023-11-27T11:07:55Z

This is amazing, does it support giving the actions any form of input?

philrenaud · 2023-11-27T17:13:48Z

@kaspergrubbe We've been toying around with the idea internally, and I'd be interested in hearing your use-case. My own feeling on the matter is that allowing operators to provide variable input for actions would create some degree of risk and make the action workflows/output less "guaranteed".

Certainly it would expand functionality and open up some cool new stuff, but we want to weigh that against the potential of providing a foot-gun here.

benvanstaveren · 2023-12-18T09:43:57Z

A late comment, but @philrenaud, I want my foot gun. I've got a few cases where being able to enter arguments to an action would be a godsend (even if triggered through the API only); perhaps cage it behind a configuration flag that needs to be explicitly enabled to allow it so that it can be decorated with the usual "warning, foot gun" disclaimers in the docs?

* Scaffolding actions (hashicorp#18639) * Task-level actions for job submissions and retrieval * FIXME: Temporary workaround to get ember dev server to pass exec through to 4646 * Update api/tasks.go Co-authored-by: Tim Gross <tgross@hashicorp.com> * Update command/agent/job_endpoint.go Co-authored-by: Tim Gross <tgross@hashicorp.com> * Diff and copy implementations * Action structs get their own file, diff updates to behave like our other diffs * Test to observe actions changes in a version update * Tests migrated into structs/diff_test and modified with PR comments in mind * APIActionToSTructsAction now returns a new value * de-comment some plain parts, remove unused action lookup * unused param in action converter --------- Co-authored-by: Tim Gross <tgross@hashicorp.com> * New endpoint: job/:id/actions (hashicorp#18690) * unused param in action converter * backing out of parse_job level and moved toward new endpoint level * Adds taskName and taskGroupName to actions at job level * Unmodified job mock actions tests * actionless job test * actionless job test * Multi group multi task actions test * HTTP method check for GET, cleaner errors in job_endpoint_test * decomment * Actions aggregated at job model level (hashicorp#18733) * Removal of temporary fix to proxy to 4646 * Run Action websocket endpoint (hashicorp#18760) * Working demo for review purposes * removal of cors passthru for websockets * Remove job_endpoint-specific ws handlers and aimed at existing alloc exec handlers instead * PR comments adressed, no need for taskGroup pass, better group and task lookups from alloc * early return in action validate and removed jobid from req args per PR comments * todo removal, we're checking later in the rpc * boolean style change on tty * Action CLI command (hashicorp#18778) * Action command init and stuck-notes * Conditional reqpath to aim at Job action endpoint * De-logged * General CLI command cleanup, observe namespace, pass action as string, get random alloc w group adherence * tab and varname cleanup * Remove action param from Allocations().Exec calls * changelog * dont nil-check acl --------- Co-authored-by: Tim Gross <tgross@hashicorp.com>

philrenaud self-assigned this Oct 18, 2023

philrenaud linked an issue Oct 18, 2023 that may be closed by this pull request

☂️ Nomad Actions #18627

Closed

32 tasks

philrenaud and others added 3 commits October 18, 2023 09:52

Actions aggregated at job model level (#18733)

3a931d1

philrenaud force-pushed the 18627-task-actions branch from d7c5e08 to 3a931d1 Compare October 18, 2023 14:11

Removal of temporary fix to proxy to 4646

95923c0

philrenaud force-pushed the 18627-task-actions branch from 67166ab to 95923c0 Compare October 18, 2023 14:15

vercel bot deployed to Preview – nomad-storybook-and-ui October 18, 2023 14:18 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui October 20, 2023 13:20 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui October 20, 2023 13:32 View deployment

changelog

bde0d8c

vercel bot deployed to Preview – nomad-storybook-and-ui October 20, 2023 14:03 View deployment

philrenaud marked this pull request as ready for review October 20, 2023 14:15

philrenaud requested a review from tgross October 20, 2023 14:15

tgross approved these changes Oct 20, 2023

View reviewed changes

nomad/job_endpoint.go Outdated Show resolved Hide resolved

dont nil-check acl

1fe0e11

vercel bot deployed to Preview – nomad-storybook-and-ui October 20, 2023 16:25 View deployment

philrenaud merged commit 8902afe into main Oct 20, 2023
28 checks passed

philrenaud deleted the 18627-task-actions branch October 20, 2023 17:05

philrenaud mentioned this pull request Oct 20, 2023

[ui] Actions implementation in the web UI #18793

Merged

schmichael mentioned this pull request Oct 31, 2023

docs: changelog & basic docs for 1.7 WI changes #18936

Merged

sevensolutions mentioned this pull request Nov 24, 2023

Custom API Signals sevensolutions/nomad-iis#27

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nomad Actions #18794

Nomad Actions #18794

philrenaud commented Oct 18, 2023 •

edited

Loading

github-actions bot commented Oct 18, 2023 •

edited

Loading

tgross left a comment

kaspergrubbe commented Nov 27, 2023

philrenaud commented Nov 27, 2023

benvanstaveren commented Dec 18, 2023

Nomad Actions #18794

Nomad Actions #18794

Conversation

philrenaud commented Oct 18, 2023 • edited Loading

github-actions bot commented Oct 18, 2023 • edited Loading

Ember Test Audit comparison

tgross left a comment

Choose a reason for hiding this comment

kaspergrubbe commented Nov 27, 2023

philrenaud commented Nov 27, 2023

benvanstaveren commented Dec 18, 2023

philrenaud commented Oct 18, 2023 •

edited

Loading

github-actions bot commented Oct 18, 2023 •

edited

Loading